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Although  mail  questionnaires  offer  economical  advantages  over  personal 
interviews,  they  often  have  lower  response  rates  which  can  offset,  at  least 
partly,  the  gains.  Yet  a  number  of  factors  have  been  shown  to  be  related 
to  improved  response  rates  to  mail  questionnaires  (Heberlein  and  Baumgartner, 
1978) .  We  examined  how  two  factors-using  special  mailing  procedures  and 
giving  monetary  incentives-might  improve  the  response  rate  to  our  mail 
questionnaires.  Both  have  proved  effective  in  earlier  projects.  Dillman 
et  al.  (1974)  found  that  certified  mail  increased  returns  in  a  follow-up 
mailing  and  House  et  al.  (1977)  showed  that  certified  mail  yields  much 
higher  returns  than  first  class  mail.  Both  studies  involved  follow-up 
mailings  whereas  the  current  report  considered  initial  mailings  to  sub¬ 
jects. 

We  studied  Navy  enlisted  men  to  whom  we  sent  questionnaires  before 
they  were  to  retire  after  twenty  years  of  service.  On  balance  our  sample 
should  have  given  us  a  high  response  rate  according  to  the  key  factors 
isolated  by  Heberlein  and  Baumgartner  (1978):  the  subjects  were  members 
of  the  armed  forces;  the  content  of  the  questionnaire  was  quite  salient 
to  the  men  since  it  dealt  with  their  impending  career  change;  and  the 
study  was  sponsored  by  a  government  organization,  the  Office  of  Naval 
Research.  On  the  other  hand,  our  design  included  two  factors  associated 
with  diminished  response  rates.  The  questionnaire  contained  attitude 
items  and  it  was  long  (it  took  about  50  minutes  to  complete) .  In  the 
initial  part  of  the  study,  we  considered  the  effects  of  incentive  payments 
and  certified  mailing  in  a  two-by-two  design.  In  later  mailings  we  re¬ 
plicated  a  portion  of  the  foregoing  and  we  tried  out  a  less  demanding 
mail  package. 

METHODS 

The  data  were  collected  as  part  of  a  longitudinal  study  of  the  adjust¬ 
ment  of  Navy  men  and  theit  wives  to  their  reentry  into  civilian  life.  The 
initial  mailing  to  the  subjects  occurred  roughly  four  to  eight  weeks  before 
they  were  scheduled  to  leave  the  Navy.  The  men  were  stationed  in  one  of 
the  fifty  states  on  shore  duty  when  they  were  sent  our  mailing.  The  initial 
mail  package  was  sent  to  the  man's  Navy  work  address.  This  demanding  package 
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contained  a  large  number  of  enclosures:  a  cover  letter  briefly  describing 
the  study  and  soliciting  cooperation,  an  additional  page  with  more  detailed 
information  about  the  study,  separate  questionnaires  for  the  man  and  his 
wife,  separate  return  envelopes  for  both  parties,  two  subject  consent  forms 
with  a  third  return  envelope.  The  man's  questionnaire  dealt  with  his  Navy 
job,  his  health,  feeling  states,  career  plans,  marriage,  coping  and  defen¬ 
sive  processes;  and  the  wife's  dealt  with  a  subset  of  the  foregoing. 

In  a  two-by-two  design,  one  half  of  the  packages  were  sent  certified 
mail  (with  no  return  receipt);  the  other  half  were  sent  regular  first-class* 
Information  about  Navy  mail  deliveries  indicated  that  an  enlisted  man  would 
almost  never  sign  for  a  package  himself  so  that  any  beneficial  effect  of 
the  certification  would  presumably  be  related  to  positive  reactions  to  the 
certified  mail  label  itself,  e.g.  the  aura  of  importance  attributed  to 
certified  mail.  One-half  of  each  of  the  above  groups  received  a  Susan  B. 
Anthony  silver  dollar  with  an  attached  note,  "This  is  a  token  of  our 
thanks  for  your  help."  The  remaining  half  of  the  above  subjects  who  were 
in  the  no  incentive  condition  did  not  receive  any  payment.  Data  from  three 
mailings  will  be  reported.  In  the  first  mailing,  the  incentive  and  mailing 
conditions  were  tested  in  the  two-by-two  design;  in  the  second,  a  partial 
replication  of  the  first,  the  incentive,  no  incentive  conditions  were  con¬ 
trasted;  and  in  the  third,  returns  of  a  pilot  questionnaire  involving  fewer 
demands  upon  the  respondents  were  examined.  These  last  respondents  had 
questionnaires  half  as  large  as  the  above;  their  wives  were  not  included; 
and  they  were  requested  to  complete  one  instead  of  three  questionnaires. 

They  also  received  the  $1  incentive. 

RESULTS 

Table  1  contains  the  results  of  the  first  two  mailings  in  the  study. 

The  returns  for  the  first  mailing  (N=*220)  show  that  the  payment  of  an  in¬ 
centive  yielded  greater  returns  than  the  absence  of  such  payments,  34.5% 
vs.  16.8%  (Chi-square®8.06,p<.01) .  On  the  other  hand,  there  was  a  minimal 
difference  between  sending  the  package  by  certified  mail  rather  than  regular 
first-class,  27.3%  vs.  24.5 %  (Chi-square-. 09,  N.S.).  The  second  mailing 
replicated  the  above  findings  of  a  greater  response  to  the  incentive  condi¬ 
tion  than  the  no-incentive  condition  (Chi-squared. 83, p<. 05)  . 

In  the  third  mailing  (N»188)  the  response  rate  was  49.5%>  significantly 
(jChi-square*1.66,p<.10;  one-tailed)  greater  than  the  largest  return  rate  in 
Table  1. 


-2- 


DISCUSSION 


In  this  study,  the  response  rate  to  a  mail  questionnaire  was  signif¬ 
icantly  greater  with  the  use  of  a  monetary  incentive  (34.5%)  as  compared 
to  a  no-incentive  condition  (16.8%).  The  finding  was  replicated  in  a  se¬ 
cond,  independent  test.  Clearly,  reimbursing  the  respondent  increases  his 
motivation  to  complete  the  questionnaire  but  the  motivation  may  not  be 
exclusively  financial  in  nature  any  more  than  our  1  token1  was  solely  in¬ 
tended  to  be  payment  for  work  to  be  done.  It  may  be  that  the  $1  establishes 
a  cooperative,  reciprocal  relationship  between  the  respondent  and  the 
survey  researcher. 

From  our  perspective,  the  experimental  manipulation  using  incentives 
was  cost-effective.  The  $1  incentive,  although  adding  little  to  the  $3 
cost  of  the  questionnaire  and  mailing  charges,  more  than  doubled  the  re¬ 
sponse  rate.  In  a  no-incentive  condition,  it  would  have  taken  twice  as 
many  mailings  at  50%  greater  cost  to  obtain  the  same  number  of  returns. 
Hence,  the  incentive  condition  resulted  in  a  savings  of  33%.  However, 
less  savings  would  result  in  studies  with  higher  initial  rates,  where  a 
ceiling  effect  might  operate,  or  where  the  questionnaire  and  mailing 
charges  were  smaller  relative  to  the  amount  of  the  incentive. 

Turning  to  mailing  procedures,  we  found  that  certified  mail,  as 
opposed  to  regular  first-class  mail,  had  no  differential  effect  upon  re¬ 
sponse  rate.  This  result  stands  in  sharp  contrast  to  the  findings  of 
Dillman  et  al.  (1974)  and  House  et  a.  (1977).  Our  result  may  be  different 
because  the  other  studies  used  certified  mail  for  follow-up  mailings 
whereas  we  used  the  same  procedure  for  the  initial  mailing.  Perhaps  the 
importance  aspect  of  certified  mail  is  more  effective  with  committed 
respondents  than  with  potential  ones.  Dillman  (1978)  argues  that  certi¬ 
fied  mail  conveys  a  sense  of  urgency  and  that  may  be  counter-productive 
with  an  Initial  request  for  help  which  should  be  low-key  in  nature.  More 
empirical  data  are  necessary  here. 

Although  the  monetary  incentive  sharply  improved  response  rates,  the 
returns  for  our  regular  questionnaire  packet  were  never  greater  than  38.6% 
of  the  number  sent  out.  Even  this  rate  is  below  the  average  response  rate 
of  48%  for  one  mailing  that  is  mentioned  by  Heberlein  and  Baumgartner  (1978), 


although  response  rates  have  declined  in  recent  years  (Steeh,  unpublished 
manuscript,  1980).  A  high  response  rate  is  desirable,  of  course,  so  that 
the  sample  can  be  representative  of  the  total  population  and  hence  the  con¬ 
clusions  can  be  generalized  beyond  the  respondents  who  returned  the  quest ion- 
naire. 

Less  than  1 %  of  our  subjects  sent  back  a  note  explicitly  refusing  to 
participate  whereas  over  8%  of  the  mailings  were  returned  because  of  in¬ 
correct  addresses.  Given  the  number  and  mobility  of  U.S.  Navy  enlisted 
personnel  it  is  likely  that  some  of  the  intended  recipients  never  received 
mailing  packets  that,  nonetheless,  were  not  subsequently  returned  to  us. 
Hence  the  percent  improvement  in  response  rate  from  the  no  incentive  condi¬ 
tion  to  the  incentive  condition  may  be  somewhat  less  than  the  100+%  change 
we  obtained. 

It  is  noteworthy  that  shortening  the  length  of  the  questionnaire  and 
reducing  other  demands  upon  the  subjects  yielded  the  most  dramatic  improve¬ 
ment  in  response  and  increased  our  rate  to  the  average  cited  earlier. 

This  substantiates  Dillman’s  (1978)  argument  that  a  group  of  procedures 
for  improving  response  rate  to  mail  questionnaires  is  the  most  practical 
approach  to  use  rather  than  relying  upon  one  technique. 


Table  1 


Percent  Response  Rate  for  Each  Mailing 
as  a  Function  of  Certified  Mail  and  Incentive  Payment 


Incentive  No  Incentive 


Certified 

1st  Mailing  38.6 

15.1 

Mail 

Not 

1st  Mailing  30.3 

18.5 

Certified 

2nd  Mailing  32. 93 

20. 03 

Means 

1st  Mailing  34.5* 

16. 81 

*Chi-square=8 . 06 , p< . 0 1 , 

one-tailed 

^Chi-square=0 . 09 ,N . S . 

3Chi-square=2 . 83 , p< . 05 , 

one-tailed 
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